Test suite prioritization by cost-based combinatorial interaction coverage

Test suite prioritization techniques modify the order in which tests within a test suite run. The goal is to order tests such that they detect faults as early as possible in the test execution cycle. Prioritization by combinatorial interaction coverage is a recent criterion that has been useful for prioritizing test suites for GUI and web applications. While studies show that this prioritization criterion can be valuable, previous studies compute the interaction coverage without considering the cost of individual tests. This paper proposes a new cost-based combinatorial interaction coverage metric, an algorithm to compute the new metric, and an empirical study with three subject web applications. Two of our studies show that prioritization by the new metric improves the rate at which faults are detected in relation to cost. A third study reveals an interesting result that the success of the cost-based metric is influenced by the distribution of t-tuples in the selected test cases.